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The purpose of this study was to determine whether third-grade teachers’ instruc- 
tional actions during reading comprehension lessons contributed to their students’ 
reading comprehension achievement. Our framework focused on teachers’ emphasis 
on three dimensions of instruction (pedagogical structure, teacher-directed instruc- 
tion, and support for student learning), as observed in comprehension lessons across 
a year. Third-grade teachers’ instruction was analyzed first by measuring their latent 
propensity to engage in instructional actions in the three dimensions and then by 
using these latent variables in a multilevel model to examine their students’ gains 
in reading comprehension. Results provided support for the theoretical dimensions, 
taking into account contextual variables including lesson, student, and teacher char- 
acteristics; teachers’ engagement in teacher-directed instruction and their support 
for student learning significantly contributed to their students’ reading compre- 
hension. Results suggest that analysis of teachers’ instructional actions within and 
across lessons is a promising approach for the study of effective reading instruction. 
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A long-standing question in educational research is the extent to which it is 
possible to identify features of teachers’ instruction that are associated with stu- 
dents’ gains in reading over the course of a year (e.g., Hoffman, 1991; Shavelson, 
Webb, & Burstein, 1986). Although past studies (including meta-analyses) have 
found that some measures of instruction have accounted for students’ achieve- 
ment gains (e.g., Brophy & Good, 1986; Seidel & Shavelson, 2007; Taylor, 
Pearson, Peterson, & Rodriguez, 2003), the current goal of ensuring that teach- 
ers are well prepared to teach students, particularly in high-poverty schools, 
has placed this question once again on center stage. As instruction tends to 
be domain specific and affected by social and organizational classroom fac- 
tors (Porter & Brophy, 1988; Seidel & Shavelson, 2007), we focus specifically 
on reading comprehension in early elementary classrooms with an interest in 
determining whether teachers’ instructional actions during lessons on reading 
comprehension relate to improvements in their students’ reading comprehension 
achievement. 

Our theoretical framework has several distinctive features. First, as just sug- 
gested, we study teachers’ instruction actions within the context of lessons on 
reading comprehension. Second, we examine instructional actions in three the- 
oretical dimensions that have been previously identified as key components of 
effective instruction. Third, because, in theory, extensiveness of use of actions 
in these dimensions is influenced by the context in which teachers teach specific 
lessons, we take features of lessons, characteristics of students, and character- 
istics of the teachers into account. Our theoretical framework and method of 
study have provided a way to embrace the complexity of reading instruction. 
In what follows, we first explain how we contextualize instruction in reading 
lessons, identify key dimensions of reading instruction, and study the relation 
of teachers’ instructional actions in these dimensions within lessons. We then 
turn to a report of the study we carried out to examine the extent to which 
teachers’ reading comprehension instruction contributes to their students’ reading 
achievement. 


A FRAMEWORK FOR ANALYSIS OF INSTRUCTION IN READING 
COMPREHENSION 

In theory, students’ comprehension development is influenced by the instruction 
in reading comprehension that the teacher provides. However, teachers adjust 
how they teach, depending on what they are teaching and to whom (Barr & 
Dreben, 1983; Stodolsky, 1990). As they plan and teach lessons, they take into 
account students’ knowledge and skills that affect their response to lessons in 
reading comprehension. This view of reading comprehension instruction reflects 
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ecological models of development, such as those proposed by Bronfenbrenner 
(e.g., Bronfenbrenner, 1979) and Sameroff (e.g., Sameroff & Friese, 2000). Based 
on these models, we see a student’s progress in learning to read in each elemen- 
tary year as dependent not just on his or her innate capabilities but also on features 
of the classroom environment to which the student is exposed, such as the knowl- 
edge of the teacher, the composition of the class, and the curriculum. In an effort 
to capture transactional aspects of teachers’ teaching and students’ learning, we 
study comprehension lessons by taking into account characteristics of students 
that might affect such instruction. 

The teacher organizes and carries out instruction — setting and clarifying the 
purpose of a lesson, organizing the classroom for instruction, selecting and car- 
rying out instructional actions and learning activities, and evaluating students’ 
progress toward learning goals. The extent to which teachers are flexible and 
adaptive in designing and carrying out reading lessons depends on their under- 
standing of reading and reading processes (e.g., linguistic, cognitive, and social 
processes involved in learning to read; Snow, Griffin, & Burns, 2005). Shulman 
(1987) described instruction as transforming the teacher’s knowledge of the 
subject area into “pedagogical representations and actions” (p. 7). He went on 
to say that “there are ways of talking, showing, enacting, or otherwise represent- 
ing ideas so that the unknowing can come to know, those without understanding 
can comprehend and discern, the unskilled can become adept” (p. 7). Connecting 
content to learners requires that teachers be flexible — that they modify their teach- 
ing to accommodate their students, the curriculum, and other relevant contextual 
factors (Bransford, Derry, Berliner, & Hammerness, 2005). Teachers’ knowledge 
about reading and reading instruction is likely to affect the nature and quality 
of instruction and students’ reading achievement (e.g., Foorman & Moats, 2004; 
McCutchen et al., 2002). 

With regard to important student characteristics, the results of previous studies 
suggest that teachers are likely to use more explicit instruction in classes with a 
high percentage of students from disadvantaged backgrounds (Connor, Morrison, 
& Petrella, 2004; Crowe, Connor, & Petscher, 2009). Although students from 
high-poverty backgrounds come to school with less well-developed language and 
literacy than their peers, reading instruction has the potential to affect their growth 
in reading, beyond that predicted by child and family variables such as poverty 
(e.g., Kainz & Vemon-Feagans, 2007). 

Other characteristics of lessons are likely to influence teachers’ instruction — 
the nature of concepts or processes being taught, the materials, the time available 
for the lesson, and so on. A question of interest is the extent to which teachers’ 
instructional actions tend to be similar across lessons, even when the content, 
materials, and activities change (see Rowan & Correnti, 2009). 
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DIMENSIONS OF READING INSTRUCTION 

Dating back to the mid-20th century, researchers have reported studies of effec- 
tive instruction that hone in on central dimensions of teachers’ instruction (e.g., 
Brophy & Good, 1986; Gage, 1978). These include the structure or overall organi- 
zation of lesson, the literacy content and how it will be delivered, and steps taken 
to ensure that students are engaged and learning (Porter & Brophy, 1988). Early 
studies of instruction often emphasized organization of the classroom as a criti- 
cal feature — that is, aspects of classroom management that contribute to students’ 
opportunities to learn (e.g., Brophy, 1986; Brophy & Good, 1986; Rosenshine 
& Stevens, 1984). They also emphasized the importance of “active teaching” 
(Brophy, 1986) and of teachers’ responsiveness to students’ needs (Rosenshine, 
1983). Assessment of these three dimensions of instruction, as observed in teach- 
ers’ lessons on reading comprehension, might distinguish more and less effective 
reading instruction. 

We refer to the three dimensions as pedagogical structure, teacher-directed 
instruction, and support for students’ learning. What follows is an explanation 
of each dimension and the instructional actions that we have selected to represent 
each one; Appendix A lists the instruction actions in each dimension and gives 
examples. 

The first of the dimensions focuses on teachers’ actions that contribute to the 
pedagogical structure (PS) of reading lessons. Although many researchers have 
seen organization as related to classroom management (e.g., having a smooth- 
running classroom with few interruptions; for example, Brophy & Good, 1986; 
Rosenshine, 1983), our focus is on actions that teachers take to help students’ 
understand the purpose and structure of a given lesson. These actions include 
providing an explanation of what the students will be learning and why and pro- 
viding clear directions for activities. In the words of Porter and Brophy (1988), 
“Effective teachers are clear about what they intend to accomplish through their 
instruction, and they keep these goals in mind both in designing the instruction 
and in communicating its purpose to the students” (p. 81). 

The second dimension is teacher-directed instruction (TD1). This dimension 
includes actions that teachers take to ensure effective learning and practice of lit- 
eracy skills and knowledge — that is, the presentation of content in a way that 
promotes learning (e.g., Brophy, 1986). Active instruction and the academic 
emphasis of instruction are similar terms used by other researchers (e.g., Brophy, 
1986; Rosenshine & Stevens, 1984). Instructional actions that focus on conveying 
literacy concepts and building students’ reading skills or processes include the 
following: providing explanations (e.g., explaining text features), modeling the 
strategies that good comprehenders use to understand texts, and providing guided 
practice (e.g., Duffy, 2002; Duffy, Roehler, & Rackliffe, 1986; Porter & Brophy, 
1988; Roehler & Duffy, 1991; Rosenshine & Stevens, 1984; Taylor et al., 2003). 
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In short, this dimension taps teachers’ understanding of how to present literacy 
content and processes to students. 

The third dimension is support for student learning (SSL). This entails actions 
on the part of the teacher to engage students in the lessons, assess their response 
to the content and activity of a lesson, and make use of students’ skills, strategies, 
and knowledge (Guthrie & Knowles, 2001; Paris & Carpenter, 2004). Effective 
teachers use instructional actions to promote students’ active involvement in lit- 
eracy tasks and to help them understand and regulate their own reading (e.g., 
Perry, VandeKamp, Mercer, & Nordby, 2002). According to Porter and Brophy 
(1988), “effective teachers continuously monitor their students’ understanding 
of presentations and responses to assignments. They routinely provide timely 
and detailed feedback, but not necessarily in the same ways for all students” 
(p. 82). Instructional actions in this dimension include providing students with 
feedback about their reading and making sure that students have opportunities to 
ask questions and contribute ideas. 

In delivering instruction and adapting planned lessons, teachers make 
choices about what to emphasize. Presumably, teachers’ instruction reflects their 
in-the-moment decisions, based on students’ response not only to the lesson but 
also to practicalities that are part of teaching in school classrooms (e.g., time 
runs out, a visitor arrives). The context is likely to affect the emphasis teach- 
ers place on each of the dimensions during a specific lesson. Studies of effective 
early reading instruction suggest that effective teachers combine teacher-directed 
instruction with guided activities for students to apply their reading and writing 
skills (e.g., Hoffman, 1991; Pressley, Rankin, & Yokoi, 1996). Similarly, McGhie- 
Richmond, Underwood, and Jordan (2007) found that teachers used instructional 
actions that characterize both transmission and constructivist styles. Similarly, 
Taylor and her colleagues (2003) found that effective teachers were likely to use 
a coaching style but at the same time structured learning activities to enhance the 
cognitive challenge for the students. 

Thus, we study comprehension instruction within the context of the lesson 
by examining teachers’ emphasis on instructional actions in each of the three 
dimensions. This approach to characterizing teachers’ reading comprehension 
instruction represents an important change, as most previous studies have exam- 
ined the effects of teachers’ behaviors on reading outcomes by counting and 
aggregating particular actions (e.g., coaching) across a day of instruction and 
sometimes across multiple days of observation. As Hoffman (1991) pointed out 
in his review of research on effective reading instruction. 


The simple counts of behaviors and their correlation to achievement outcomes have 
yielded an enormous amount of data but not much insight into teaching, school- 
ing and learning. The lens of science has been focused on such a small area 
that the meaningfulness of the behaviors observed is lost without reference to the 
surrounding context, (p. 945) 
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Like Hoffman, we see the need for studies of teachers’ comprehension instruction 
that take a more nuanced approach than the use of simple aggregates or averages 
of behaviors to focus on dimensions believed to be central to effective instruction 
within the context of lessons on reading comprehension. 


HOW TO STUDY INSTRUCTION IN COMPREHENSION LESSONS? 

Our approach to studying effective instruction in teachers’ reading comprehen- 
sion lessons involved two steps. We first studied instructional actions within 
dimensions as they were present in observed comprehension lessons on 4 days 
of observation of the literacy block across a year. We define lessons as discrete 
events with the purpose of teaching reading comprehension to a given group. 
First, to take into account each teacher’s emphasis on each of the three dimensions 
within lessons, we carried out a multilevel, multidimensional analysis. That is, we 
embedded instructional actions within dimensions in the context of each lesson, 
taking into account lesson features (e.g., duration) and including characteristics 
of the class (e.g., presence of students who are English language learners) and 
the teacher (e.g., teachers’ performance on a test of reading knowledge). The sec- 
ond step involved using the latent variable representing each teacher’s instruction 
in each dimension (the results from the aforementioned analysis) to examine the 
extent to which these contributed to students’ reading achievement. This approach 
addressed problems encountered by other researchers seeking to determine why 
some teachers’ instruction is more effective than others. 

One such study is Taylor et al. (2003); these researchers examined aspects 
of instruction that fostered higher level thinking, encouraged independent use of 
word reading and comprehension strategies, provided student support, and pro- 
moted students’ active involvement in literacy tasks. Their observation system 
involved coding of 19 variables related to instruction; the researchers examined 
the influence of content variables (e.g., phonics) and instructional actions (e.g., 
telling or modeling) on students’ reading achievement. Results showed that higher 
order questions and modeling contributed to students’ growth in reading in Grades 
2 to 5. An important finding was that many of the variables related to student 
outcomes were infrequently observed, and this included higher order questions. 
For instance, modeling was observed in 3 to 5% of the segments. The researchers 
wisely pointed out that the relation of frequency of occurrence and effects on long- 
term growth needed further examination. We address this issue through examining 
instructional actions within dimensions, as they are used in lessons on reading 
comprehension. 

Other studies have analyzed data from systematic observations by summing 
the presence of certain features across the entire time spent observing instruction 
and then carrying out factor analysis (or similar techniques) to characterize 
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aspects of teachers’ literacy instruction (e.g., Foorman et al., 2006; Pianta, Belsky, 
Vandergrift, Houts, & Morrison, 2008). The purpose is often to collect infor- 
mation on literacy instruction, in general (e.g., purposes, activities, products), 
without a specific focus on teachers’ instructional actions. In Foorman et al. 
(2006), the observation measure used time sampling to record 8 categories for 
instructional format (e.g., whole class) and 20 categories for content (e.g., word 
work, previewing a text before reading). The content categories included both lit- 
eracy content/purpose (e.g., vocabulary) and pedagogical features (e.g., giving 
directions and preparing for instruction). The data were analyzed by summing 
the frequencies of the time allocation variables across all of the observations 
(four times a year) for each teacher. A principal components analysis yielded 
seven components that included a mix of different instructional and content 
characteristics. However, the researchers found that the time allocation compo- 
nents alone did not significantly account for students’ achievement gains. One 
possible explanation is that differences in instruction in particular literacy areas 
(e.g., phonics, vocabulary) are likely to be obscured when they are averaged across 
all lessons and observations. 

Pianta and his colleagues (2008) studied the relation of emotional and instruc- 
tional support and student’s reading achievement in a large-scale, longitudinal 
study. Their observations involved time-sampling, during which observers kept 
notes which informed their global ratings of emotional and instructional support 
in first-, third-, and fifth-grade classrooms; global ratings were based on a set of 
7-point rating scales. Global ratings at the classroom level that focused on instruc- 
tion included productive use of time (e.g., efficient transitions) and richness of 
instructional methods (e.g., encouragement of discussion). Two factor compos- 
ites (emotional support and instructional support) were examined to determine the 
extent to which they contributed to students’ growth in reading over time. Results 
showed that the ratings of instructional support did not account for variance in 
first, third, or fifth graders’ gains in reading achievement. 1 The researchers did 
find significant effects of emotional support at all three grade levels. The results 
showed the importance of teachers’ sensitivity to students’ needs and confirmed 
our decision that support for student learning was a critical dimension of teach- 
ers’ instruction, although teacher behaviors that represented this dimension in our 
framework were more related to support for learning than support for emotional 
well-being. 

Connor and her colleagues used a different approach to examine the extent to 
which the content emphasis and management of instruction (teacher managed vs. 
student managed) were related to students’ reading and language outcomes (e.g., 
Connor et al., 2004; Connor, Morrison, & Underwood, 2008). In Connor et al. 


1 Similar findings are reported in Hamre and Pianta (2005) and Pianta, Belsky, Houts, and Morrison 
(2007). 
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(2008), results showed that students who started first grade with little knowl- 
edge of word reading made stronger gains by the end of second grade if they 
spent more time in teacher-managed, code-focused instruction. Because Connor 
and colleagues focused on the appropriateness of instruction for individual stu- 
dents, they collapsed instructional activities into two categories (teacher managed 
or student managed). As our purpose is to understand the effects of teachers’ 
instructional actions on students’ reading comprehension, we grouped instruc- 
tional actions within dimensions in reading comprehension lessons. However, 
like Connor et al. (2008) we examine students’ prior achievement as a factor in 
teachers’ emphasis on particular dimensions of instruction. 

To summarize, our framework breaks new ground by using lesson as a unit 
of analysis, by focusing on teachers’ instructional actions in three theoretical 
dimensions, and by using methods of data analysis that take multiple influences 
on instructional actions into account. We carried out a multivariate, multidi- 
mensional measurement model that made it possible to examine together the 
instructional actions within the three dimensions of reading instruction; variation 
in teaching as it occurs over the course of the school year; and various features 
of students, classrooms, and schools that shape reading instruction. Contextual 
variables that might influence how teachers teach reading comprehension were 
taken into account; these included teachers’ degree attainment and performance 
on a measure of teacher knowledge (Snow et al., 2005) and information about 
students aggregated at the level of the classroom (e.g., prior reading achieve- 
ment, English language status). We addressed two research questions: How well 
can we measure each theoretical dimension of comprehension instruction? Which 
characteristics of classrooms (teachers and students) are associated with reading 
instruction characterized by each of the three dimensions? 

This done, we used the results of the measurement model to investigate the 
extent to which each teacher’s relative emphasis on each of the three dimensions 
within lessons, as previously described (a latent variable), was associated with 
student achievement. The research questions for the achievement model were 
as follows: To what extent did teachers’ observed instructional actions along 
each of the scaled dimensions (estimated by the measurement model) account 
for students’ reading comprehension performance? To what extent did teachers’ 
instructional actions modify the relationship between student characteristics (e.g., 
poverty indicators) and achievement in reading comprehension? We describe the 
method of the study and then present each of these analyses in turn. 

METHOD 


Participants 

The data for this study were drawn from a larger study of instruction in second- 
and third-grade classrooms in Reading First (RF) schools in a midwestern state. 
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Schools participating in RF were characterized by high poverty and a history of 
low reading achievement. We recruited six districts with RF schools and carried 
out four observations of the literacy block of 88 second- and third-grade teachers 
in 19 schools in these districts. For the present study, we focus on the 44 teachers 
who taught third grade. Of these, 91% were female, 21% were non- White, and 
52% had a master’s degree. On average, these teachers had more than 13 years of 
teaching experience (range = 1-39 years). 

The average class we observed had 22 students (range = 13-28). Roughly 36% 
of the students in the classroom were racial minority students (non- White and 
non-Asian; range = 0-100%). Eligibility for free or reduced price lunch (FRL) in 
the classrooms ranged from 12% to 100% with a mean of 73%; on average, 18% 
of the students were of limited English proficiency (range = 0-100%). 

We encountered a small amount of missing data at both the teacher and stu- 
dent levels (approximately 2% at the teacher level and 12% at the student level). 
Although unverifiable directly, our missing data analyses suggested that there was 
no relation between missing data and either achievement or instructional practice. 
Rather than remove cases with incomplete data, we employed the missing at ran- 
dom (Rubin, 1976) assumption and used multiple imputation to address missing 
data (e.g., Raghunathan, Lepkowski, Van Hoewyk, & Solenberger, 2001). 

Observation Data 

Automated Classroom Observation System for Reading is a system designed to 
code the presence of features of early elementary reading instruction. The data 
collection system involved a program on a PC tablet with a number of automatic 
coding and data management features. Coding was carried out in 5-min intervals; 
during that interval, the observer recorded the following fields: purpose of the 
lesson, the grouping arrangement (e.g., whole class), materials used in the les- 
son, instructional actions, word meaning actions, and average number of students 
actively engaged in the lesson. The observer could designate a change in activity 
to indicate the start of a new lesson within a 5-min interval. Observers used text 
boxes to record notes about the content of instruction. 

After two training sessions and opportunities to compare observers’ coding 
of videotaped lessons, we assigned two observers to carry out observations in 
each participating school. On an initial visit to a classroom (and one other time 
during the year), the observers coded the instruction for the entire literacy block 
(90 min to 2 hr) independently. We assessed interobserver reliability in two ways. 
The first was agreement on the number of lessons and the purpose of each lesson. 
Across six pairs of observers (one of the 1 1 observers is a member of two observer 
pairs), overall agreement was 88%. Second, we compared the agreement of code 
assignment across all fields and all options within each field. Overall agreement 
was 87.2% with a range of 80% to 96% agreement. 
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In the first stage of data analysis, the literacy block was divided into lessons, 
which were identified by purpose (e.g., phonics) and grouping (i.e., if phonics 
lessons were taught to more than one group, each was considered a separate 
lesson). Time for each lesson was compiled by summing over intervals and subin- 
tervals. In selecting lessons for this study, we included comprehension lessons 
that involved texts and included lessons taught to the whole class, a small group, 
or an individual. These selection criteria yielded 287 comprehension lessons that 
constituted about 27% of the total number of third-grade lessons (accounting for 
23% of the instructional time). (For purposes of comparison, 8.7% of all the 
lessons were phonics; 12% were vocabulary; 29% were guided reading/ small- 
group lessons.) The average duration of the comprehension lessons was 19.3 
min (17.07 SD). Analysis of the proportion of lessons in which each of the 
instructional actions in the three dimensions (as shown in Table 1) shows that 
TDI actions were present in a high proportion of lessons; more variable were the 
actions in PS and SSL. 


TABLE 1 

Proportion of Instructional Action in the Three Dimensions in Comprehension Lessons 


Theoretical Dimension 

Instructional Action 

Proportion of Lessons 
Action Was Used* 

Pedagogical structure (PS) 

PS 1 : Explaining the purpose of 
the lesson 

0.36 


PS2: Explaining the 
value/relevance of the 
lesson 

0.09 


PS3: Giving directions for 
activity 

0.78 


PS4: Providing a wrap up or 
summary of what has been 
accomplished 

0.13 

Teacher-directed instruction (TDI) 

TDIl: Telling 

0.77 


TDI2: Modeling/ Coaching 

0.61 


TDI3: Asking questions for 
evaluation 

0.85 


TDM: Providing practice or 
review activities 

0.69 

Support for student learning (SSL) 

SSL1: Fostering discussion 

0.29 


SSL2: Assessing students’ 
work; providing feedback 

0.22 


SSL3: Gives students an 
opportunity to ask questions 

0.10 


a n = 287. 
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Teachers’ Knowledge and Professional Background 

Measures of teacher’s educational background and experience were included 
because of their potential to be related to a teacher’s decisions about how to teach 
reading comprehension. These included the area of bachelor degree specializa- 
tion, the attainment of a master’s degree, the area of master’s degree, and the 
number of years a teacher had taught previously. We also included data from sur- 
veys that teachers had completed while participating in the evaluation of RF. One 
survey, administered in the spring, was a measure of teachers’ knowledge, entitled 
Teachers’ Knowledge of Reading and Reading Practices (TKRRP); the measure 
is made up of 22 items focused on teachers’ knowledge about reading (Carlisle, 
Johnson, Phelps, & Rowan, 2008). The measure includes items of foundational 
skills (e.g., phonemic awareness) as well as comprehension. The scale had an 
item response theory reliability of 0.76. 

Second, teachers completed surveys about their instructional practices. One 
focused on the extent to which teachers designed instruction to meet the needs of 
students in their classroom. The survey items are provided in Appendix B. Three 
additional survey items provided relevant information on teachers’ practice: “In 
this school year, about how often have you followed the sequence of instruction 
suggested in the comprehensive reading program?” (hereafter, Comprehensive 
reading program usage); “In this school year, about how often have you pro- 
vided explicit instruction (e.g., modeling, demonstrations, examples)?” (hereafter, 
Explicit instruction usage); and “In this school year, about how often have you 
used student data to guide lesson planning?” (hereafter, Student data usage). 
These self-reported practices provided balance to TKRRP, which assessed content 
knowledge embedded in classroom scenarios. 

Students’ Reading Achievement 

The Iowa Tests of Basic Skills (ITBS) reading comprehension standard score 
was used as the measure of students’ reading achievement. This test requires 
students to select responses to questions that follow short passages. Test reli- 
ability for the reading comprehension subtest in Grade 3 was 0.91 (computed 
with Kuder-Richardson Formula 20; Hoover, Dunbar, & Frisbe, 2003). In addi- 
tion, three measures were used to control for prior reading achievement: the ITBS 
Comprehension subtest from the previous two years and their fall Oral Reading 
Fluency (ORF) score from the Dynamic Indicators of Basic Literacy (DIBELS; 
https://dibels.uoregon.edu). 

Measurement Model Method 

The purposes of the measurement model were (a) to summarize each teacher’s 
enacted instruction in reading comprehension lessons along three theoretical 
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dimensions and (b) to examine the extent to which classroom and teacher and 
student characteristics were related to teachers’ instruction. We were particu- 
larly interested in the extent to which teachers’ knowledge was related to each 
dimension. 


Data analysis. This analysis was guided by several assumptions. The first is 
that teachers’ propensities to employ various instructional actions are shaped by 
various contexts that can change across different observations across the school 
year. The second is that the instructional actions measure different, distinct dimen- 
sions of instruction. The third is that although the dimensions are distinct, they 
are interrelated and influence one another. These considerations, combined with 
the nested nature of the data (i.e., lessons grouped within teachers), led us to 
employ a multivariate, multilevel Rasch measurement model (Kelcey, Carlisle, 
Berebitsky, & Phelps, 2009). To characterize teachers’ enacted instruction, we 
used the observed instructional actions to estimate each teacher’s latent location 
on each of the three dimensions. 

This approach has certain advantages over other methods of statistical analysis, 
such as summing occurrences of instructional action across a day’s observation. 
First, the multivariate component enables modeling of the interrelations between 
the instructional actions that comprise each of the hypothesized dimensions to 
measure practice. Thus, emphasis of any one dimension is dependent on the 
emphasis of the other dimensions, controlling for various classroom and school 
variables. Second, the use of random effects addresses the dependency among 
instructional actions within a lesson and a teacher. Third, a multilevel structure 
allows us to assess the variation in instructional actions among classrooms and 
lessons, as well as the association of various student, classroom, and school char- 
acteristics with the instructional actions. The multilevel structure also includes 
a control for the time of year the lesson was taught, thus taking into account 
how instruction can vary across a school year. Finally, the measurement aspect 
of our approach provides a way to understand the use of instructional actions 
within lessons for each teacher along an interval scale on each of the hypothe- 
sized dimensions. We are able to model how instructional actions interact with 
each other, the characteristics of a lesson, and characteristics of the teachers and 
students (Kelcey et al., 2009). 

The model consisted of three structured levels. The first level was a mea- 
surement model, describing how the instructional actions were related to the 
hypothesized dimensions. The second, or lesson, level addressed the depen- 
dency among instructional actions within a lesson and examined the relationship 
between instructional actions and characteristics of the lesson. We explored vari- 
ables at the lesson level to control for the context in which the instruction 
occurred, including the duration of the lesson and the number of days since the 
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beginning of the year that the observation occurred. The third level represented 
teachers and classrooms, addressing the lack of independence among lessons 
within a teacher while exploring the relationship between the instructional actions 
and characteristics of the teachers and their classrooms. The teacher variables 
included responses to survey questions that provided self-report about reading 
instruction practices. In addition, we tested the extent to which teachers’ knowl- 
edge was related to each of the instructional dimensions. As this analysis involved 
multiple hypothesis tests (teacher knowledge on each dimension), we adjusted the 
significance levels using a sequential Bonferroni correction (e.g., Abdi, 2007). 
We controlled for measures of the student composition of the class by aggregat- 
ing student demographic data. These were the percentage of minority students 
(non- White and non-Asian), the percentage of students qualifying for FRL, the 
percentage of students designated as special education, the percent of students 
designated as Limited English Proficient (LEP), and the number of students in the 
classroom. 

To review, the measurement model generated a summary of each teacher’s 
level of enacted instruction for each dimension. These summaries (or latent 
scores) provided an index of the teachers’ emphasis on the instructional actions 
in that dimension, controlling for classroom and school variables. These latent 
scores were then used to assess the relation of enacted practice with students’ 
achievement. 

The measurement model equation is as follows: 

Level 1: 

4 3 

^lijk ijk(jT-T!)I pjk T ^ ' &TDI mjk^TDImijk) T ^SSLijki^-SSLjk T ^ ® SSLmjk^SSLmijk ) 

m= 1 m= 1 


4 

+ D PSi j k (n PS j k + ^ CUpSmjkOpSmijk ) ( 1 ) 

m— 1 

where D is an indicator taking on a value of 1 if the ith item is in the scale that 
measures practice trait p, 0 otherwise 

Ttpjiu is the log odds of employing an instructional action in observation j in teacher 
k to the reference item within practice trait type p 
a pmij k= 1 if item i is the mth item within scale p, 0 otherwise 
“pmjk is the discrepancy between the log odds of employing an instructional action 
for the mth item in scale p for teacher k and the reference item within that scale, 
holding constant 7r p jk 

Level 2: 

2 

J*TDIjk — Ao TDIk + As TDIk^sk + u TDIjk 
s= 1 
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S= 1 
2 



ctpmjk — Upmkforp = TDI,SSL,PS',m = 1 , ... 10 


(2) 


Level 3: 



n= 1 
N 



(3) 


Achievement Model Methods 

The second phase of the data analysis focused on the question of whether each of 
the measures of teachers’ enacted instruction accounted for significant variation 
in students’ reading comprehension, when taking into account various student- 
and teacher-level variables, including several measures of prior achievement. 
Previous research has shown that the effectiveness of teachers and their instruc- 
tional practices vary by students’ socioeconomic economic status (SES) and prior 
achievement (e.g., Nye, Konstantopoulos, & Hedges, 2004). In particular, there 
tends to be more variation and larger teacher effects among low SES or low- 
achieving classrooms than high SES or high-achieving classrooms. Accordingly, 
we examined the extent to which teachers’ instructional actions in reading com- 
prehension modified the relationship of students’ achievement with both prior 
achievement and their eligibility for FRL. 

Data analysis. To estimate the association of teachers’ instructional 
approaches with student achievement, we modeled student performance on ITBS 
Reading Comprehension, using a hierarchical random intercept and slope lin- 
ear model (e.g., Raudenbush & Bryk, 2002). Accordingly, we retained nine null 
hypotheses (i.e., three measures of practice on the intercept and three on each of 
two slopes — one being prior achievement and one being FRL). As in the mea- 
surement model, the use of multiple hypotheses required the use of a sequential 
Bonferroni correction to adjust the significance levels (e.g., Abdi, 2007). 

All independent Level 1 and Level 2 variables were centered around their 
respective grand means. At Level 1, we included seven student covariates. The 
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first two variables were allowed to vary randomly and their slopes were modeled: 
(jt | ) represented eligibility for FRL and (it 2) represented the spring 2007 ITBS 
reading comprehension score. The final five variables included both demographic 
dummy variables and prior achievement continuous measures: (jt 3 ) indicated 
minority status (non- White and non-Asian), ( 774 ) indicated LEP, (its) indicated 
special education status, (jto) represented the fall 2007 ORF score, and (ttj) rep- 
resented the spring 2006 ITBS reading comprehension score . 2 The general form 
of the model at Level 1 was 


n= 7 

Yjj = JT 0 j + 'y' JTpXpij + Sij, (4) 

P= 1 


where Y is the 2008 ITBS reading comprehension score, representing the appro- 
priate reading outcome, noj is the average student score adjusted for the student 
variables, X, and tc p are the variable’s corresponding coefficients, whereas e y has 
a normal distribution with mean zero and variance a 2 . At Level 2, we modeled 
the adjusted average, rcoj, using teacher and classroom variables. Before testing 
our hypotheses, we adjusted for relevant imbalances on teacher-level characteris- 
tics, using forward stepwise regression. The control variables we considered were 
the same set of teacher experience and education measures and survey responses, 
including the measure of teacher knowledge, used in the measurement model. 

The form of the model at Level 2 was 

Q 

noj = Poo + PoqWqj + PoQ+iPSj + fioQ+T-YDIj + Poq+^SSLj + rq/ 

9=1 


Q 

K\j = P10 + P\qWqj + P\Q + \PSj + Pxq+jTDIj + PiQ + ?,SSLj + r y 

q=l 


Q 

1*2] = P 20 + Pi qWqj + PiQ+\PSj + PlQ+lTDIj + PlQ+lSSLj + Ty, (5) 

9=1 


where Poo is the average adjusted achievement for a teacher’s class; P q o is average 
effect of covariate W ql on adjusted achievement; and PS j, TDI ; , and SSL are the 


2 For the dummy variables, the reference group represented students not eligible for free/reduced 
price lunch. White or Asian students, English proficient students, and students not designated as 
special education. 
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teachers’ latent locations on the three dimensions of instruction with correspond- 
ing coefficients P-q+i t° P-q+3- Moreover, rj is the random effect of teacher j and 
has a normal distribution with mean zero and variance x n . 


RESULTS 


Measurement Model 

A key assumption in our analysis was that the instructional actions measure dif- 
ferent dimensions of instruction. We investigated this assumption by assessing 
the estimated covariance structure of the model. If the dimensions correlated very 
close to 1 or —1, then the constructs would be considered identical. The mag- 
nitude of the correlations of pairs of the three dimensions (shown in Table 2) 
indicates that the dimensions are distinguishable. Furthermore, examination of 
the variance decomposition between the lesson and teacher levels for each of the 
three dimensions (also shown in Table 2) provides support for the assumption that 
the dimensions are distinct. Across all three dimensions, there was more variation 
at the lesson than the teacher level, but the range of variation indicated that the 
dimensions were different from each other. 

We also studied whether the instructional actions were appropriately grouped 
into the three dimensions. In a Rasch model, each action should have a similar 
slope. By visually analyzing the item characteristic curves and utilizing statistical 
tests (likelihood ratio test and Bayesian Information Criterion), we established 
that the instructional actions in both the TDI and SSL dimensions were well 
grouped. However, one action in the PS dimension, “Provides a wrap-up or sum- 
mary,” did not align with the others. We dropped this instructional action from all 
analyses because it could not consistently discriminate in a way that was similar 
to the other instructional actions in that dimension. 


TABLE 2 

Correlations Between Instructional Dimensions at the Teacher Level and Variance 
Components of Instructional Dimensions 



Correlations Between 
Dimensions 

Unconditional Variance 
Components 

Final Variance Components 

TDI 

SSL PS 

t jr (Lesson) 

Tp ( Teacher ) 

Xn (Lesson) 

x p (Teacher) 

TDI 

— 


1.37* 

0.73* 

0.72* 

0.09* 

SSL 

-0.07 

— 

1.42* 

0.42* 

0.99* 

0.05 

PS 

-0.48 

0.18 — 

3.68* 

1.12* 

2.65* 

0.09 


Note. TDI = teacher-directed instruction; SSL = support for student teaming; PS = pedagogical 
structure. 

*p < .05. 
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TABLE 3 

Results of the Multivariate, Multilevel Rasch Measurement Model 




Coefficient (SE) 



TDI 

SSL 

PS 

Teacher level 

Intercept 

0.13(1.56) 

0.52(1.71) 

6.60 (2.28)* 

Master’s 

0.52 (0.22)* 

-0.36 (0.24) 

0.19(0.31) 

Teacher knowledge 

-0.30 (0.12)* 

0.44(0.14)* 

0.41 (0.17)* 

Student data usage 

0.32 (0.15)* 

-0.26 (0.17) 

-0.04 (0.22) 

Comprehensive reading program usage 

-0.14(0.26) 

0.12(0.27) 

-0.98 (0.36)* 

Explicit instruction usage 

0.98 (0.44)* 

-1.20 (0.47)* 

-0.12(0.62) 

% free/reduced lunch 

-3.09 (0.49)* 

1.18(0.50)* 

-0.04 (0.64) 

% minority 

1.52 (0.29)* 

— 1.15 (0.33)* 

-1.67 (0.42)* 

Average prior ability 

-0.03 (0.01)* 

0.04(0.01)* 

-0.01 (0.01) 

Lesson level 

Lesson duration 

0.06 (0.01)* 

0.03 (0.01)* 

0.07 (0.01)* 

Observation period 

0.00 (0.00) 

0.00 (0.00) 

0.00 (0.00) 


Note. TDI = teacher-directed instruction; SSL = support for student learning; PS = pedagogical 
structure. 

*p = .05. For the Teacher Knowledge measure, the asterisk indicates significance at the p = .05 
level after adjusting p values using a sequential Bonferroni correction. 


In carrying out the measurement model, we examined the relation of a number 
of lesson-level and teacher-level variables and the three theoretical dimensions 
of reading instruction (see Table 3). With regard to characteristics of lessons, 
results showed that lesson duration positively influenced the propensity of using 
instructional actions in each of the three dimensions. As might be expected, 
the longer a lesson, the more opportunity there was to carry out instructional 
actions. However, the time of year in which the observation took place was not 
significantly associated with any of the latent dimensions. 

With regard to teacher characteristics and knowledge, results showed that 
teacher knowledge had a significant association with all three dimensions; how- 
ever, the direction of the association was not constant across the three dimensions. 
A teacher with a higher teacher knowledge score was more likely to emphasize PS 
and SSL but less likely to engage in TDI. Having a master’s degree increased the 
likelihood that a teacher emphasized TDI. With regard to teachers’ self-reported 
practices, teachers who were more likely to follow closely the comprehensive 
reading program were less likely to provide PS for their students; in contrast, 
this variable was not correlated with either of the other dimensions. Teachers 
who reported frequently using explicit instruction on the survey items were more 
likely to use more instructional actions in the TDI dimension and less likely to 
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provide SSL. Similarly, teachers who reported frequent use of student assessments 
to guide lessons tended to use more TDI. 

At the teacher level, the percentage of minority students in the class was associ- 
ated with each of the three instructional dimensions, albeit not always in the same 
way. A classroom with more minority students increased the likelihood of the use 
of TDI while decreasing the likelihood of providing PS or SSL. A higher propor- 
tion of minority students in the classroom increased the odds that a teacher would 
use TDI and decreased the odds that a teacher would provide SSL. On the other 
hand, the percentage of FRL students had the reverse effect: classes with higher 
percentages of FRL students were less likely to have lessons that emphasized TDI 
and more likely to have a teacher who emphasized SSL. Finally, in classes with 
a higher average entering ability (as measured by mean class performance on the 
fall ORF DIBELS subtest), teachers had a lower probability of using TDI but a 
higher probability of emphasizing SSL. 

The measurement model explained fairly large portions of the variability in 
teachers’ instructional actions. At the teacher level, the contextual model was 
able to explain roughly 90% of the variance in each dimension of instructional 
practice. Similarly, but to a lesser extent, the contextual model for the variation in 
instruction among lessons was able to explain roughly 30 to 50% of the variation, 
depending on the dimension (see Table 3). 

Achievement Model 

Using fully unconditional models, we partitioned the variance in students’ ITBS 
reading comprehension outcome into two components, representing the variance 
between teachers and between students within classrooms. 

As shown in Table 4, the intraclass correlations indicate that the majority of 
variation in student achievement was attributed to the students within classrooms 
(83.9%). Variation in this component often reflects such factors as natural apti- 
tude, motivation, and family support, but it also includes measurement error. The 
teacher level contributed the remainder of the variation (16.1%). 


TABLE 4 

Variance Components for the Achievement Model 


Component 

Final Model 

Unconditional Model 

Intercept Reliability (X) 

Teachers (r) 

0.003 

70.60 

0.79 

Students (s) 

169.05 

367.77 

— 

ICC 

<.01 

0.16 



Note. ICC = intraclass correlations. 
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The estimates of our final achievement model, including the three dimensions 
of instruction, are shown in Table 5. Results showed that, as expected, students 
who had higher scores on all three measures of prior achievement (2007 ITBS 
comprehension, 2006 ITBS comprehension, and fall 2007 DIBELS ORF) per- 
formed better on the 2008 ITBS comprehension test. At the student level, the 
results indicated that LEP students scored lower on average than their peers. 

The indices of teachers’ enacted instruction for two of the dimensions (TDI 
and SSL) were significantly and positively associated with student achievement 
in reading comprehension. Holding other factors constant, a 1 standard deviation 
increase in a teacher’s use of TDI was associated with a 0.17 standard deviation 
gain in students’ reading comprehension achievement over a 1-year period. The 
effect size for SSL was smaller; a 1 standard deviation increase in SSL usage 
was associated with a 0.08 standard deviation increase in student achievement. 
In addition, two self-reports of teachers’ instructional practice showed positive 
association with student’s achievement: frequency of use of the comprehensive 
program and of differentiated instruction. When the teacher had a master’s degree 
specializing in reading/literacy, students had, on average, higher achievement on 
ITBS reading comprehension, after controlling for all other variables in the model. 

In examining the results of the slope models, we found that two of the 
instructional dimensions significantly modified the relationship between FRL and 
achievement. In particular, higher levels of both TDI and SSL tended to increase 
reading comprehension achievement more for students eligible for FRL than for 
students not eligible for FRL. These results are shown in Figure 1 for TDI and 
Figure 2 for SSL. 

Teachers’ knowledge on TKRRP also modified the association between FRL 
and student achievement. The results suggest that higher knowledge teachers 
were, on average, more effective with students not eligible for FRL. The rela- 
tionship between the 2007 ITBS comprehension slope and performance on 2008 
ITBS reading comprehension was also significantly modified by a teacher’s posi- 
tion on both the latent TDI and SSL dimensions. Students with higher scores on 
their pretest tended to perform better on the assessment if they were in classrooms 
with teachers who used more TDI and SSL. The same pattern held for usage of 
the comprehensive program: Students scoring higher on the pre-test received a 
boost when the teacher said he or she followed the program. 

Of additional interest is the residual variation at the teacher level, after adjust- 
ing for variables included in the final model (reported in Table 3). The model did 
a good job of explaining variability in instruction among teachers but less so for 
the variability in instruction among lessons. The variation at Level 2 was reduced 
from a significant value of 70.6 to an insignificant level ip > .05) of 0.003. Almost 
all of the variation at Level 2 was explained, which suggests that the covariates 
included in this model leave very little additional variation to be explained by 
unmeasured covariates. However, our sample size is small, and it is likely that 
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TABLE 5 

Results of the Achievement Model 

Coefficient (SE) 

Intercept 

-0.01 

(0.05) 

Years teaching experience 

0.00 

( 0 . 00 ) 

Master’s: Reading/Literacy 

0.31 

(0.13)* 

Teacher knowledge 

0.02 

(0.04) 

Comprehensive reading program usage 

0.13 

(0.04) 

Differentiated instruction 

0.10 

(0.03)* 

TDI 

0.17 

(0.05)* 

SSL 

0.08 

(0.06)* 

PS 

0.04 

(0.05) 

FRL status (n i ) 



Intercept 

-0.09 

(0.13) 

Years teaching experience 

- 0.01 

( 0 . 01 ) 

Master’s: Reading/Literacy 

0.03 

(0.30) 

Teacher knowledge 

-0.23 

(0.08)* 

Comprehensive reading program usage 

0.11 

(0.09) 

Differentiated instruction 

-0.14 

(0.08) 

TDI 

0.42 

( 0 . 12 )* 

SSL 

0.38 

(0.13)* 

PS 

0.20 

( 0 . 10 ) 

ITBS 2007 RC score ( 7 r 2 ) 



Intercept 

0.19 

(0.05)* 

Years teaching experience 

- 0.00 

( 0 . 00 ) 

Master’s: Reading/Literacy 

0.22 

(0.14) 

Teacher knowledge 

-0.03 

(0.03) 

Comprehensive reading program usage 

0.10 

(0.04)* 

Differentiated instruction 

-0.03 

(0.03) 

TDI 

0.15 

(0.04)* 

SSL 

0.15 

(0.05)* 

PS 

0.05 

(0.04) 

Minority (^ 3 ) 

-0.13 

(0.09) 

Limited English proficient {nf) 

-0.19 

(0.08)* 

Special education (^ 5 ) 

- 0.10 

(0.08) 

Fall 2007 ORF score (ne) 

0.39 

(0.03)* 

ITBS 2006 RC score (tzj) 

0.07 

(0.03)* 


Note. TDI = teacher-directed instruction; SSL = support for stu- 
dent learning; PS = pedagogical structure; FRL = free or reduced 
price lunch; ITBS = Iowa Tests of Basic Skills; RC = reading com- 
prehension; ORF = Oral Reading Fluency. 

*p > .05. For the TDI, SSL, and PS measures, the askerisk indi- 
cates significance at the p = .05 level after adjusting p values using 
a sequential Bonferroni correction. 
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FIGURE 1 Interaction of reading comprehension achievement and teacher-directed instruction 
(TDI) emphasis by students’ eligibility for free or reduced price lunch (FRL). Note. The TDI values 
are standardized coefficients. 
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FIGURE 2 Interaction of reading comprehension achievement and support for student learning 
(SSL) emphasis by students’ eligibility for free or reduced price lunch (FRL). Note. The SSL values 
are standardized coefficients. 


with a larger sample size, the variation in teachers’ instructional actions would be 
greater. 


DISCUSSION 

The purpose of this study was to determine whether teachers’ instructional actions 
in teaching reading comprehension lessons to their third graders contributed 
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to their students’ improvement in reading comprehension across the year. The 
study was motivated by the general perception among educators and researchers 
that teachers’ instruction (both what and how they teach) affects their students’ 
academic progress. Few studies have identified instructional practices in early lit- 
eracy that contribute to students’ gains in reading, although few researchers have 
focused specifically on the ways that teachers select and combine instructional 
actions in teaching specific areas of literacy. In most other studies of teachers’ 
instruction, judgments of quality of instruction, not teachers’ actual instructional 
actions, have been found to explain variance in students’ progress in reading (e.g., 
Cirino, Pollard-Durodola, Foorman, Carlson, & Francis, 2007; Hamre & Pianta, 
2005; Pianta et al„ 2008). 

We focused specifically on observed teachers’ instructional actions that rep- 
resent three critical dimensions of instruction — PS (e.g., clarifying the purpose 
of a lesson), TDI (e.g., modeling), and SSL (e.g., fostering discussion). The 
results suggest that teachers’ emphasis on two of the three dimensions (TDI 
and SSL) accounted for significant variance in students’ reading comprehension 
achievement. Further, results confirmed several key expectations, such as a signif- 
icant relation between teachers’ knowledge and emphasis on actions in the three 
dimensions. Overall, the findings suggest that our framework and analytic meth- 
ods offer a promising approach to identifying the instructional actions effective 
elementary teachers use when they teach lessons in specific areas, such as reading 
comprehension. 

Analysis of Reading Comprehension Instruction 

Our first step was development of a theoretical framework that contextualized 
teachers’ instructional actions within lessons on reading comprehension. In early 
elementary classrooms, the lesson is a natural unit of instruction, as teachers 
prepare for and teach lessons that have a particular goal, using particular mate- 
rials and methods of instruction (e.g., Stodolsky, 1990). The three dimensions 
of instruction we chose to study (PS, TDI, and SSL) were derived from previ- 
ous studies of effective early reading instruction. Our observations of instruction 
included the entire literacy block in 44 third-grade classrooms four times during 
the year. From these, we identified 287 comprehension lessons taught by these 
teachers; these constituted about one fourth of all their lessons. 

Our first step involved examining instructional actions within each dimen- 
sion in these lessons, taking into account the influence of lesson features, student 
characteristics, and teachers’ characteristics. The analysis was a multilevel, multi- 
dimensional measurement model that yielded a number of interesting and relevant 
findings. First, results suggest that we could measure each of the three dimensions 
with a set of indicators that fit each scale. We also found that within each dimen- 
sion, all instructional actions discriminated similarly with the exception of one 
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instructional action under PS (“Providing a wrap-up of the lesson”); this item was 
eliminated from the analyses in order to construct a proper Rasch scale. Second, 
at the lesson level, lesson duration contributed significantly to the extensive use of 
instructional actions in each dimension. This stands to reason, as longer lessons 
provide more opportunity to use a variety of methods. One finding at this level 
that we had not anticipated was that the time of the observation did not contribute 
significantly, suggesting that lessons taught early in the year did not differ signif- 
icantly from those later in the year. Although this result suggests that the ways 
each teacher taught comprehension lessons was reasonably stable, others have 
found significant variation among occurrences (e.g., Rowan & Correnti, 2009). 
This, then, is an important topic for further study. 

Third, at the teacher level, we found that teacher knowledge contributed to each 
dimension. This contribution was positive for SSL and PS but negative for TDI; 
the negative relation of teacher knowledge and TDI is perplexing. One possible 
explanation is that instructional actions in the TDI dimension (e.g., modeling) are 
prevalent for both knowledgeable and less knowledgeable teachers but that more 
knowledgeable teachers are less likely to emphasize actions in the TDI dimension. 
A second possible explanation is that less knowledgeable teachers might work 
in higher poverty contexts, where TDI dimensions are more likely to be empha- 
sized. A Some researchers have suggested that teachers’ content knowledge about 
reading contributes to their use of explicit instruction (e.g., Foorman & Moats, 
2004), whereas others have suggested that knowledgeable teachers are less reliant 
on specific teaching routines, which might include instructional actions in the TDI 
dimension (e.g., Bransford et al., 2005). 

Another important finding was that certain characteristics of students in their 
class affected teachers’ instruction. This is somewhat similar to findings reported 
by Harare and Pianta (2005), which indicated that quality of reading instruction 
contributed to narrowing the achievement gap for students at risk. We found that 
teachers tended to use more instructional actions in the TDI dimension and fewer 
in SSL and PS dimensions when there were more minority students in their class. 
With regard to the extensive use of TDI, Smith, Lee, and Newmann (2001) found 
that didactic instruction (roughly comparable to TDI) was more frequently used in 
schools with high percentages of minority students. These researchers also found 
that a higher percentage of time providing interactive learning experiences was 
also associated with significant gains in students’ reading. These results suggest 
the value of examining different dimensions of instruction concurrently in specific 
types of lessons, rather than isolating particular instructional methods for study, 
especially if the goal is overall profiles of effective instruction (e.g., Pressley et al., 
2001 ). 

With regard to variation in instructional actions, our findings were like those of 
other researchers (e.g., Pianta et al., 2007; Rowan & Correnti, 2009; Taylor et al., 
2003) in showing extreme variability in the use of particular instructional actions, 
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as was apparent in Table 1 . Our results are also similar to others in the relative 
scarcity of certain instructional actions, such as fostering discussion (e.g., Pianta 
et al., 2008). For example, in the PS dimension, teachers fairly often explained 
the purpose of the lesson (present in 36% of the lessons) but seldom explained 
the value of the literacy goal or activity to the students (observed in 9% of the 
lessons). Teachers frequently used instructional actions in the TDI dimension 
(e.g., modeling and coaching 61%) but less frequently used instructional actions 
in the SSL dimension (2.g., fostering discussion, 29%). One value of the measure- 
ment model was that it provided a way to capture teachers’ relative preferences 
for using frequent and infrequent actions in different lesson contexts. 

Comprehension Instruction and Students’ Gains in Reading 

The major finding of our study is that third-grade teachers’ relative emphasis on 
instruction in two of the three dimensions (TDI and SSL) contributed signifi- 
cantly to students’ gains in reading achievement across the year, when accounting 
for prior achievement and both teacher and student characteristics. To put these 
findings in perspective, recall that in recent studies, time-sampled variables and 
their aggregates related to teachers’ instruction have not accounted for significant 
variance in their students’ reading achievement (e.g., Foorman et al., 2006; Pianta 
et al., 2008). For example, Pianta et al. (2008) found that there was no effect for 
quantity of exposure to reading instruction or instructional support on first, third, 
or fifth graders’ reading gains. This result seems counterintuitive, as in theory 
teachers’ instructional actions ought to affect their students’ reading achieve- 
ment. For this reason, it was gratifying to find that teachers’ emphasis on two 
dimensions of instruction in comprehension lessons, taking into account con- 
text and teacher and student characteristics, did contribute to students’ reading 
comprehension achievement. 

These findings are encouraging and suggest that our analytic approach holds 
promise for future study of effective elementary reading instruction. Along with 
this approach, other features of the study might have contributed to the out- 
come. One is the alignment of purpose of instruction and students’ outcome, both 
focused on reading comprehension (e.g., Shavelson et al., 1986). A second fea- 
ture is the focus on instructional actions that might be specific to literacy prevalent 
in the early elementary years; the need to study “curriculum specific” aspects of 
instruction was noted by Pianta et al. (2008) as one shortcoming of their observa- 
tion system. A third feature is the focus on theoretical dimensions of instruction 
in reading comprehension. Because the measurement model yielded latent indices 
of enacted instruction for each of the three dimensions for each teacher, we were 
not faced with the problem Taylor et al. (2003) encountered; specifically, they 
found that higher level questions accounted significantly for student gains in read- 
ing, but because such questions were very infrequently observed, the researchers 
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found it hard to explain how these very uncommon questions could have had such 
a substantial effect on students’ reading achievement. 

It is important to recognize that teachers’ instruction was represented in all 
three dimensions; these are not entirely independent of one another. The signif- 
icant negative correlation of TDI and PS suggests that a teacher who tended to 
emphasize actions in the TDI dimension (e.g., telling, modeling) was unlikely to 
emphasize actions in the PS dimension (e.g., explaining the purpose of a lesson). 
It would be a mistake to think that the dimensions represent independent cate- 
gories of teaching styles. Unlike Smith et al. (2001), our model does not classify 
teachers as a TDI, PS, or SSL teacher; instead, it quantifies the unique blend of 
instructional strategies used by each teacher. 

From the achievement model another important finding is that certain teacher 
and student characteristics significantly affected reading outcomes, whereas oth- 
ers did not. With regard to teachers’ knowledge, we found that the effect of 
teachers’ knowledge on students’ achievement was different from what we found 
in the measurement model. That is, whereas teachers’ knowledge about reading 
significantly contributed to the three dimensions, it did not explain students’ gains 
in reading over the year. This combination of findings suggests the importance of 
knowledge about reading in designing and carrying out comprehension lessons 
while not significantly contributing additionally to overall gains on a standard- 
ized reading achievement. Other studies we have carried out have (e.g., Carlisle, 
Correnti, Phelps, & Zeng, 2009) also found performance on a general measure 
of knowledge about reading to be weakly related to student outcomes, so the 
results of the present study reinforce the view that it is knowledge in practice that 
contributes to students’ progress in reading. 

With regard to student characteristics, we found that modeling the slopes of 
both the percentage of students qualifying for subsidized lunch (FRL) and the 
performance on the prior year’s comprehension assessment provided additional 
insight into the influence of instruction on achievement. As Figures 1 and 2 show, 
FRL students benefited more than their non-FRL peers from being in classrooms 
where teachers were particularly invested in using TDI and providing SSL. The 
findings indicate that teachers’ emphasis on instructional actions in the TDI and 
SSL dimensions was not equally effective for all students. This finding is compat- 
ible with the results of other studies (e.g., Connor et al., 2008) that suggest that 
students at higher achievement levels need less direction and focused explanation 
from teachers. 

Limitations and Directions for Further Study 

Our analyses of teachers’ instructional actions in comprehension lessons have 
provided results that do show that what teachers do when they teach compre- 
hension affects their students’ progress in reading comprehension. Still, because 
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of the limitations of the study, the major contribution might be the theoretical 
framework and the application of advanced statistical methods. For purposes of 
this initial study of our framework and analytic methods, we focused solely on 
comprehension lessons taught by third-grade teachers. However, we know that 
the nature and quality of instruction is likely to vary by area of reading (Barr & 
Dreeben, 1983), and so it will be important to apply this framework to lessons in 
other areas (e.g., phonics) and at other grade levels. Further research might also 
focus on the contextual variables that affect outcomes of reading lessons, not only 
student and teacher characteristics but also types of materials used in lessons (e.g., 
trade books, basal reading programs). 

Our study has other limitations: the sample of teachers is small and drawn from 
relatively high-poverty, underachieving schools in one state. The targeted instruc- 
tional actions were intentionally limited in number, leaving lots of room to explore 
others. Nonetheless, we do not think that there could ever be such a thing as a set 
of instructional actions that all teachers should use in all comprehension lessons 
for all students — and further research is needed to test this perspective. Certainly, 
we are in the beginning stages of the work we have undertaken to understand 
the relation of teachers’ instructional actions and the effects these have on their 
students’ reading performance. 
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APPENDIX A 

The Three Dimensions, as Represented by Specific Items and Examples 



The teacher: 

For example: 

Pedagogical structure 

Explains purpose of lesson. 

“Today we are going to learn how to 
use the table of contents in a 
book.” 


Explains value / relevance of lesson 

“The table of contents is a very 


(e.g., how it will be useful in the 

helpful way to find information. 


future). 

You will be using the Table of 
Contents in many books as you do 
research for your science reports.” 


Gives directions for activity. 

“Now that we have looked at the title 
and pictures, talk with your 
partner about what you already 
know about tides. Take notes to 
share with the class.” 


Provides a wrap-up or summary a 

“So, let’s go over the steps in writing 


(reviews what the group has worked 
on, what has been learned, 
sometimes including feedback 
directed toward the group about 
progress in mastering a concept). 

a summary one last time.” 

Teacher-directed 

Tells / explains 

The teacher might explain a 

instruction 

(includes explaining ideas, giving 

procedure or strategy (e.g., how to 


information, and providing 

summarize information in a 


explicit instruction). 

passage) or provide background 
information 


Models 1 coaches. 

Modeling the process of using 


(Modeling occurs when the teacher 

context to infer a word’s meaning 


shows the students how to carry 

(e.g., “Persuade. I don’t know 


out a procedure or a way of 

what that means. Hmmm. Let’s 


thinking, often explaining his/her 

see if there are clues in the 


thinking while carrying out 

sentence. . . .”). Coaching: 


(Continued) 
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APPENDIX A 

(Continued) 


The teacher: 


For example: 


Support for Student 
Learning 


out the procedure. Coaching 
occurs when the teacher gives the 
students suggestions, clues, or 
reminders.) 


Asks questions for evaluation (asking 
questions for which the teacher 
knows the answer in order to 
assess students’ understanding and 
provide feedback of some kind to 
student responses). 

Provides practice or review 
activities. 

Fosters or initiates discussion 
(initiates discussion and engages 
students’ thinking, providing them 
with opportunities to express their 
own ideas or their interpretations 
of a text). 

Assesses students’ learning; provides 
feedback (assesses students’ 
learning or performance and 
provides feedback for individual 
students. Feedback is typically 
based on the teacher’s analysis of 
the student’s performance in that 
lesson and might include 
suggestions for improvement or 
brief reteaching). 

Gives students opportunity to ask 
questions (offers students 
opportunities to contribute ideas 
or ask questions; indicates that 
students might have problems 
understanding a concept, 
procedure, or activity or might 
have ideas or questions about the 
content). 


‘Remember how you found clues to 
infer word meanings in the story 
we read yesterday? See if you can 
use the same process to 
understand that word that you are 
struggling with today.” 

‘Who is the main character? Where 
does the story take place?” 


The teacher might share ideas, 
remarks, or queries (e.g., “oh, 
what a good idea”). Such 
comments are used to monitor or 
extend the discussion, not to 
evaluate students’ responses 
“Jean, I can understand that you are 
interested in the topic of the book 
you have chosen to read, but this 
book is too easy for you. Perhaps 
you could find another book that 
will be more challenging for 
you — one that will keep you 
interested.” 


“I can tell from your expression, 
Maria, that you are confused. 
What did I say or what did your 
classmates say about the meaning 
of that word that confused you?” 


‘As explained in the text, this item was deleted from the model. 
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APPENDIX B 

Survey Item Stems Included in the Models 

In this school year, about how often have you . . . 

• used student data to guide lesson planning? 

• followed the sequence of instruction suggested in the comprehensive reading 
program? 

• provided explicit instruction (e.g., modeling, demonstrations, examples)? 

Differentiated Instruction Item Stems 

• I make efforts to recognize all students’ individual progress in reading. 

• I provide several activities in class so that students can choose from among 
them. 

• I use a wide range of assignments, materials, or activities matched to students’ 
needs and skill levels. 

• I use a wide range of assignments, materials, or activities matched to students’ 
interests. 

• I use flexible grouping in my classroom. 

• I frequently use assessments to decide what my students need next. 

• I am able to design instruction to meet the learning needs of all my students. 

• I regularly offer students opportunities to choose learning activities, individuals 
to work with, or books to read. 


